Classification techniques are widely used to manage and organize large volumes of scattered data into meaningful categories. In particular, classification is the task of analyzing numerical properties of various features depicted in the data, and organizing them into categories. This categorization of data into classes can be helpful in many applications, including computer-aided medical diagnosis, treatment effectiveness analysis, performance prediction, marketing and even financial analysis.
Classification is a form of learning that is based on the assumption that the data in question depicts one or more features, and that each of these features belongs to one or several distinct and exclusive classes. In particular, classification typically involves generating a model (or classifier) based on a training set of data samples accompanied by class labels. During the training phase, characteristic properties of typical features are isolated and, based on these features a classifier that uniquely describes the classification category is generated.
Many methods may be used to train the classifier, such as regression trees and Adaboost. Such methods, however, typically aim to decrease training error in a greedy manner. Since such greedy algorithms always make the immediate locally optimal decision at each node or phase, they tend to converge to local maxima or plateaus. The greedy learning process cannot guarantee a globally optimal solution, especially in multi-modal datasets that include many local maxima.
As such, it is desirable to provide a more effective method that minimizes classification error and achieves a more globally optimal solution.